Monte Carlo methods Week 3 , Markov chain Monte

Markov chain Monte Carlo, or MCMC, is a way to sample probability distributions that cannot be sampled practically using direct samplers. This includes a majority of probability distributions of practical interest. MCMC runs a Markov chain X1, X2, . . ., where Xk+1 is computed from Xk and some other i.i.d. random input. From a coding point of view, a direct solver is X = fSamp();, while the MCMC sampler is X = fSamp(X);. Both the direct and MCMC samplers can call uSamp(); many times. The Markov chain is designed so that the probability distribution of Xk converges to the desired distribution f . It turns out to be possible to create suitable practical Markov chains for many distributions that do not have practical direct samplers. Two theorems underly the application of MCMC, the Perron Frobenius theorem, and the ergodic theorem for Markov chains. Perron Frobenius says, among other things, if a “reasonable” Markov chain preserves the distribution f , then the distribution of Xk converges to f as k → ∞. The chain preserves f if Xk ∼ f =⇒ Xk+1 ∼ f . The theorem says that a “reasonable” chain that preserves automatically converges to f . It is easy to design reasonable chains that preserve f . Any such chain will sample f . Of course, we don’t want just one sample, we want a large number of samples that can do things like estimate